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CIRCUIT AND METHOD FOR GENERATING FILLER PIXELS FROM THE 
ORIGINAL PIXELS IN A VIDEO STREAM 

Technical Field: 

5 The invention relates generally to electronic circuits, and more particularly to a 

circuit and method for estimating the values of filler pixels from the values of original 
pixels in a video stream. For example, one can use the circuit or method to de- 
interlace an interlaced video stream. That is, from the original pixels in the original 
video fields, one can generate the values of filler pixels, form complementary filler 

1 0 video fields from the filler pixels, and combine the original and complementary fields 
to generate respective video frames. 

Background of the Invention: 

Figure 1 is a diagram of an interlaced video frame 10, which includes an even 
15 video field 12 and an odd video field 14. The even field 12 includes the even lines 
aO, a2, a4 . . . a(n-1) of the frame 10, and the odd field 14 includes the odd lines b1 , 
b3, b5 . . . bn of the frame 10. A video source (not shown) such as a video camera 
generates the even field 12 at a time to and generates the odd field 14 at a 
subsequent time ti, and a video display (not shown in Figure 1) displays the fields 12 
20 and 14 in the same sequence. For example, according to the National Television 
Standards Committee (NTSC) video standard, which has been in existence for over 
50 years, a video source generates and a video display displays one field 12 or 14 
every 1/60 th of a second, and thus respectively generates and displays one frame 10 
every 1/30 th of a second. But even though the display displays the fields 12 and 14 
25 at different times, the relatively slow frequency responses of the display and the 

human eye cause a viewer to perceive that the display is displaying the fields 12 and 
14 simultaneously. Thus, the viewer perceives the frame 14 as a single image 
instead of two sequential partial images. 

Many modem video applications, however, generate streams of non- 
30 interlaced, i.e., progressive, video frames. For example, most applications of the 
new High-Definition Television (HDTV) standards such as MPEG (Motion Pictures 
Experts Group) call for the generation and display of an entire frame 14 
approximately every 1/60 th of a second. Because such MPEG video sources and 
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displays respectively generate and display all the lines of a progressive frame at one 
time and not at two sequential times, progressive frames contain little if any motion 
blurring. 

Because many existing video sources generate interlaced video, and because 
5 many existing video works are recorded in an interlaced format, one may wish to 
convert a stream of interlaced video frames into a stream of progressive video 
frames that are compatible with HDTV systems. For example, one may wish to 
convert a VHS signal from a VCR (not shown) into a progressive video signal for 
display on an HDTV display (not shown in Figure 1). 
10 Still referring to Figure 1 , a simple technique for de-interlacing the interlaced 

frame 10 is to merge the fields 12 and 14 into a resulting progressive frame that is 
displayed twice in a row at the frame-display rate. For example, in the MPEG 
standard described above, a display displays this resulting progressive frame and 
then displays it again 1 /60 th of a second later. But because the fields 12 and 14 
1 5 were generated at different times t 0 and ti , the resulting progressive frame may 

contain blurred regions, particularly if there were changes in the image contents, i.e., 
motion, between the times to and ti. Thus unfortunately, this technique often results 
in a video stream of relatively poor visual quality by HDTV standards. 

Another technique for de-interlacing the video frame 10 is to generate 
20 respective complimentary filler fields for the original fields 12 and 14. That is, for the 
even field 12, one "fills" the missing odd lines with odd filler lines, and for the odd 
field 14, one fills the missing even lines with even filler lines. 

One approach to this filler technique is to spatially interpolate the filler pixels 
of the filler lines from the values of neighboring original pixels within the same field. 
25 This approach is typically most accurate when there is significant motion between 
the original fields. Unfortunately, many spatial interpolation approaches have a 
tendency to falsely interpolate a thin line as a directional edge, and thus introduce 
artifacts into the resulting progressive frame. 

An alternative approach is to temporally interpolate the values of the filler 
30 pixels from the values of corresponding original pixels in adjacent complimentary 
original fields. This approach is typically most accurate when there is little or no 
motion between the original fields. 
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Because many interlaced video streams have some segments that exhibit 
significant inter-field motion and other segments that exhibit little or no inter-field 
motion, another approach, often called a hybrid approach, combines the spatial and 
temporal interpolation approaches. For example, one hybrid approach varies the 
5 relative weightings of the temporal and spatial interpolation approaches based on the 
magnitude of inter-field motion. The greater the magnitude of inter-field motion, the 
more heavily weighted the spatial interpolation approach; conversely, the lower the 
magnitude of inter-field motion, the more heavily weighted the temporal interpolation 
approach. 

1 0 Unfortunately, many hybrid techniques sometimes fail to detect significant 

inter-field motion, and thus assign improper weightings to the spatial and temporal 
approaches. Although some of these techniques can be modified to overcome this 
defect, such modifications often require an impractical amount of memory. 

1 5 Overview of Conventional Image-Compression Techniques 

To help the reader more easily understand the concepts discussed below in 
the description of the invention, the following is a basic overview of the relevant 
aspects of conventional image-compression techniques. 

To electronically transmit a relatively high-resolution image over a relatively 

20 low-band-width channel, or to electronically store such an image in a relatively small 
memory space, it is often necessary to compress the digital data that represents the 
image. Such image compression typically involves reducing the number of data bits 
necessary to represent an image. For example, High-Definition-Television (HDTV) 
video images are compressed to allow their transmission over existing television 

25 channels. Without compression, HDTV video images would require transmission 
channels having bandwidths much greater than the bandwidths of existing television 
channels. Or, to reduce data traffic and transmission time to acceptable levels, one 
may compress an image before sending it over the internet. In addition, to increase 
the image-storage capacity of a CD-ROM or server, on may compress an image 

30 before storing it. 

Referring to Figures 2A - 3D, the basics of the popular block-based MPEG 
compression standards, which include MPEG-1 and MPEG-2, are discussed. 
Figures 2A - 2D illustrate compressing a Y-C B -C R image (e.g., video frames or fields) 
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according to an MPEG 4:2:0 format, and Figures 3A - 3D illustrate compressing a Y- 
Cb-Cr image according to an MPEG 4:2:2 format. But the discussed concepts also 
apply to other MPEG formats, to images that are represented in other color spaces, 
and to other block-based compression standards such as the Joint Photographic 

5 Experts Group (JPEG) standard, which is often used to compress still images. 
Although many details of the MPEG standards and the Y, C B , C R color space are 
omitted for brevity, these details are well-known and are disclosed in a large number 
of available references including "Video Compression" by Peter D. Symes, McGraw- 
Hill, 1998, which is incorporated by reference. Furthermore, other well-known block- 

1 0 based compression techniques are available for encoding and decoding video and 
still images. 

Referring to Figures 2A - 2D, the MPEG standards are often used to 
compress temporal sequences of images — video frames for purposes of this 
discussion — such as found in a television broadcast. Each video frame is divided 

15 into subregions called macro blocks, which each include one or more pixels. Figure 
2A is a 1 6-pixel-by-1 6-pixel macro block 20 having 256 pixels 22 (not drawn to 
scale). In the MPEG standards, a macro block is always 16x16 pixels, although 
other compression standards may use macro blocks having other dimensions. In the 
original video frame, i.e., the frame before compression, each pixel 22 has a 

20 respective luminance value Y and a respective pair of color-, i.e., chroma-, difference 
values Cb and Cr. 

Before compression of the video frame, the digital luminance (Y) and chroma- 
difference (C B and C R ) values that will be used for compression, i.e., the original or 
pre-compression values, are generated from the original Y, C B , and C R values of the 

25 original frame. In the MPEG 4:2:0 format, the pre-compression Y values are the 
same as the original Y values. Thus, each pixel 22 merely retains its original 
luminance value Y. But to reduce the amount of data to be compressed, the MPEG 
4:2:0 format allows only one pre-compression Cb value and one pre-compression Cr 
value for each group 24 of four pixels 22. Each of these pre-compression C B and C R 

30 values are respectively derived from the original C B and C R values of the four pixels 
22 in the respective group 24. For example, a pre-compression C B value may equal 
the average of the original C B values of the four pixels 22 in the respective group 24. 
Thus, referring to Figures 2B - 2D, the pre-compression Y, C B , and C R values 
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generated for the macro block 20 are arranged as one 16x16 matrix 26 of pre- 
compression Y values (equal to the original Y values for each respective pixel 22), 
one 8x8 matrix 28 of pre-compression C B values (equal to one derived C B value for 
each group 24 of four pixels 22), and one 8 x 8 matrix 30 of pre-compression C R 
5 values (equal to one derived C R value for each group 24 of four pixels 22). The 
matrices 26, 28, and 30 are often called "blocks" of values. Furthermore, because it 
is convenient to perform the compression transforms on 8 x 8 blocks of pixel values 
instead of on 16 x 16 blocks, the block 26 of pre-compression Y values is subdivided 
into four 8x8 blocks 32a - 32d, which respectively correspond to the 8x8 blocks A 
10 - D of pixels 22 in the macro block 20. Thus, referring to Figures 2A - 2D, six 8 x 8 
blocks of pre-compression pixel data are generated for each macro block 20: four 8 x 
8 blocks 32a - 32d of pre-compression Y values, one 8 x 8 block 28 of pre- 
compression C B values, and one 8x8 block 30 of pre-compression C R values. 
Figures 3A - 3D illustrate the generation of the pre-compression Y, C B , C R , 
1 5 values according to the MPEG 4:2:2 format. Referring to Figure 3A, the pixels 22 of 
the macro block 20 are arranged in two-pixel groups 34 as compared to the four- 
pixel groups 24 (Figure 2A) that the 4:2:0 format calls for. Referring to Figure 3B, in 
the MPEG 4:2:2 format, the pre-compression Y values are the same as the original Y 
values. Thus, as in the 4:2:0 format, each pixel 22 merely retains its original 
20 luminance value Y. But referring to Figures 3C and 3D, to reduce the amount of 

data to be compressed, the MPEG 4:2:2 format allows only one pre-compression C B 
value and one pre-compression Cr value for each group 34 of two pixels 22. Each 
of these pre-compression C B and C R values are respectively derived from the original 
C B and C R values of the two pixels 22 in the respective group 34. For example, a 
25 pre-compression C B value may equal the average of the original C B values of the two 
pixels 22 in the respective group 34. Therefore the 4:2:2 format calls for twice as 
many C B and C R values (one per every two original pixels) as the 4:2:0 format (one 
per every four pixels). Thus, referring to Figures 3B - 3D, the pre-compression Y, 
C B , and C R values generated for the macro block 20 of Figure 3A are arranged as 
30 one 1 6 x 1 6 matrix 36 of pre-compression Y values (equal to the original Y values for 
each respective pixel 22), one 8x16 matrix 38 of pre-compression C B values (equal 
to one derived C B value for each group 34 of two pixels 22), and one 8x16 matrix 
40 of pre-compression C R values (equal to one derived C R value for each group 34 
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of two pixels 22). As discussed above, because it is convenient to perform the 
compression transforms on 8 x 8 blocks of pixel values instead of 16 x 16 or 8 x 16 
blocks, the block 36 of pre-compression Y values is subdivided into four 8x8 blocks 
42a - 42d, which respectively correspond to the 8 x 8 blocks A - D of pixels in the 

5 macro block 20. Likewise, the block 38 of pre-compression C B values is subdivided 
into two 8 x 8 blocks 44a and 44b, which correspond to the pairs of blocks A and B 
and C and D, respectively. Similarly, the block 40 of pre-compression C R values is 
subdivided into two 8 x 8 blocks 46a and 46b, which correspond to the pairs of 
blocks A and B and C and D, respectively. Thus, referring to Figures 3A - 3D, eight 

10 8x8 blocks of pre-compression pixel data are generated for each macro block 20: 
. four 8x8 blocks 42a - 42d of pre-compression Y values, two 8 x 8 blocks 44a - 44b 
of pre-compression C B values, and two 8 x 8 blocks 46a - 46a of pre-compression 
Cr values. 

1 5 SUMMARY OF THE INVENTION 

In one aspect of the invention, an image processing circuit includes a 
processor that receives a value of an original pixel of an original first video image 
and a value of an original pixel of an original second video image. The processor 
generates a first pixel-value component from the value of the original pixel of the first 
20 original video image, and generates a second pixel-value component from the value 
of the original pixel in the original second video image. From the first and second 
pixel-value components, the processor generates a value of a filler pixel, and 
combines the filler pixel and the original first video image to generate a resulting 
video image. 

25 One can use such an image processing circuit to generate a filler video field 

from an original video field and to merge the filler and original fields to generate a 
resulting video frame. Such an image processing circuit often uses less memory and 
detects inter-field motion more accurately than prior image processing circuits. 
In another aspect of the invention, the processor of the image processing 

30 circuit receives first and second sets of pixel values for first and second respective 
groups of original pixels in an original video image. The processor calculates 
direction values from the first and second sets of pixel values for a filler pixel that is 
for disposition in the original video image between the first and second groups of 
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original pixels. The processor generates a value for the filler pixel based on the 
calculated direction values. 

One can use such an image processing circuit to spatially interpolate the filler- 
pixels of a filler field from the original pixels in an original field and to merge the filler 
5 and original fields to generate a resulting video frame. Such an image processing 
circuit often distinguishes thin lines from edges more accurately, arid thus often 
produces fewer visual artifacts, than prior image processing circuits. 

BRIEF DESCRIPTION OF THE DRAWINGS 

10 Figure 1 is an interlaced video frame according to the prior art. 

Figure 2A is a diagram of a macro block of pixels that are arranged in 2 x 2 
groups according to a conventional MPEG 4:2:0 format 

Figure 2B is a diagram of a block of pre-compression luminance values that 
respectively correspond to the pixels in the macro block of Figure 2A according to a 
1 5 conventional MPEG 4:2:0 format. 

Figures 2C and 2D are diagrams of blocks of pre-compression chrominance 
values that respectively correspond to the pixel groups in the macro block of Figure 
2A according to a conventional MPEG 4:2:0 format. 

Figure 3A is a diagram of a macro block of pixels that are arranged in 2 x 1 
20 groups according to a conventional MPEG 4:2:2 format. 

Figure 3B is a diagram of a block of pre-compression luminance values that 
respectively correspond to the pixels in the macro block of Figure 3A according to a 
conventional MPEG 4:2:2 format. 

Figures 3C and 3D are diagrams of blocks of pre-compression chrominance 
25 values that respectively correspond to the pixel groups in the macro block of Figure 
3A according to a conventional MPEG 4:2:2 format. 

Figure 4 is a block diagram of an image processing circuit according to an 
embodiment of the invention. 

Figure 5 is a flow diagram showing the general operation of the image 
30 processing circuit of Figure 4 according to an embodiment of the invention. 

Figure 6 is a timing diagram of a sequence of video fields according to an 
embodiment of the invention. 
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Figure 7 is a diagram of two consecutive 4:2:0-formatted even video fields 
from Figure 6 and their respective odd filler fields according to an embodiment of the 
invention. 

Figure 8 is a plot of the transfer function of a raw-motion-value filter according 
to an embodiment of the invention. 

Figure 9 is a motion-value buffer for the filler fields of Figures 7 and 1 1 
according to an embodiment of the invention. 

Figure 10 is a motion-trace buffer for the filler fields of Figures 7 and 1 1 
according to an embodiment of the invention. 

Figure 1 1 is a diagram of two consecutive 4:2:0-formatted odd video fields 
from Figure 6 and their respective even filler fields according to an embodiment of 
the invention. 

Figure 12 is a diagram of two consecutive 4:2:2-formatted even video fields 
from Figure 6 and their respective odd filler fields according to an embodiment of the 
invention. 

Figure 13 is a motion-value buffer for the filler fields of Figures 12 and 15 
according to an embodiment of the invention. 

Figure 14 is a motion-trace buffer for the filler fields of Figures 12 and 15 
according to an embodiment of the invention. 

Figure 15 is a diagram of two consecutive 4:2:2-formatted odd video fields 
from Figure 6 and their respective even filler fields according to an embodiment of 
the invention. 

Figure 16 is a flow diagram of a technique for loading and updating the 
contents of the motion-value buffers of Figures 9 and 12 and the motion-trace buffers 
of Figures 10 and 13 according to an embodiment of the invention. 

Figure 17 is a diagram of the original pixels used to calculate direction vectors 
and spatially interpolated pixel values for a filler pixel according to an embodiment of 
the invention. 

Figures 18A - 16E illustrate the possible direction vectors for the filler pixel of 
Figure 17 according to an embodiment of the invention. 
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DETAILED DESCRIPTION OF THE INVENTION 

Figure 5 is a block diagram of an image processing circuit 50 according to an 
embodiment of the invention. The circuit 50 can calculate the values of filler pixels 
from the original pixels in the original video fields, generate filler fields from the filler 
5 pixels, and merge the filler fields and original fields to generate resulting video 
frames. In one embodiment, the circuit 50 extends the influence of detected inter- 
field motion to more filler fields than prior image processing circuits. Thus, the circuit 
50 often generates more accurate filler fields and higher-quality resulting frames than 
prior circuits. In another embodiment, the circuit 50 uses less memory for storing 
10 motion information than many prior circuits. In yet another embodiment, the circuit 
50 spatially interpolates thin lines and edges more accurately than many prior 
circuits, and this further increases the accuracy of the filler fields and the visual 
quality of the resulting frames. 

The image processing circuit 50 includes a field buffer 52, which receives and 
1 5 stores one or more original video fields from a stream of original video fields. A 
processor unit 54 includes a processor 56 and a memory 58, generates filler fields 
from the original fields stored in the buffer 52, and merges the filler and original fields 
to generate respective resulting video frames. A frame buffer 60 stores the frames 
generated by the unit 54 and provides them for display on a video display 62. In one 
20 embodiment, the processor 56 is a Very Long Instruction Word (VLIW) processor 
manufactured by Equator Technologies of Seattle, WA. In another embodiment, the 
unit 54 generates the resulting frames in a HDTV format and the display 62 is an 
HDTV display. In yet another embodiment, a high-capacity storage device such as a 
Digital Video Disk (DVD) replaces the frame buffer 60 and stores the resulting video 
25 frames for later display. 

Referring to Figure 5, the operation of the image processing circuit 50 of 
Figure 4 according to an embodiment of the invention is discussed in conjunction 
with the flow diagram 63. 

Referring to block 64 of the flow diagram 63, the processor unit 54 retrieves 
30 from the field buffer 52 the values of the original pixels that respectively compose the 
next two original non-complimentary video fields in the sequence of original video 
fields. 
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Next, referring to block 66, from the retrieved values of the original pixels the 
processor 56 calculates a motion value for a group of filler pixels. This group 
includes one or more filler pixels that the processor 56 will merge with the first of the 
two non-complimentary original fields to form a resulting frame. 
5 Referring to block 68, the processor 56 spatially interpolates a respective pixel 

value for each of the filler pixels in the group. 

Referring to block 70, the processor 56 also temporally interpolates a 
respective pixel value for each filler pixel in the group. 

Next, referring to block 72, the processor 56 calculates respective spatial and 
1 0 temporal weighting factors from the motion value and weights the spatially and 

temporally interpolated pixel values with these respective factors. The processor 56 
then combines these weighted pixel values to generate a respective resulting pixel 
value for each filler pixel in the group. The processor unit 54 store these resulting 
filler pixel values in the frame buffer 60, or stores them in the memory 58 until the 
1 5 processor 56 generates the entire filler field . 

Referring to block 74, if the processor 56 must generate more filler-pixel 
values to complete the respective filler field, then the processor unit 54 returns to 
block 66. Conversely, referring to block 76, if the processor 56 has completed the 
current filler field but there are more original fields in the buffer 52, then the 
20 processor unit 54 returns to block 64. But if there are no more original fields in the 
field buffer 52, then the processor unit 54 halts filler-field generation. 

Referring to Figures 6 - 18E, the steps of the flow diagram 63 of Figure 5 are 
discussed in more detail according to an embodiment of the invention. 

Figure 6 is a timing diagram of a sequence of video fields that the field buffer 
25 52 of Figure 4 receives according to an embodiment of the invention. The sequence 
includes four even fields E 0 - E 3 , which alternate with four odd fields O 0 - 0 3 for a 
total of eight fields, i.e., four interlaced frames. Each field occurs at a relative time t 
within the sequence. For example, the odd field 0 3 is the most recent field at time t 7 , 
and the even field E 0 is the least recent field at time to. Furthermore, the original 
30 lines of each field are shown as closed blocks, and the filler lines of the 

complimentary filler fields are shown as open blocks. For example, the original field 
E 0 includes original even lines aO, a2, a4, a(k-1), and the odd filler field that the 
image processing circuit 50 of Figure 4 will generate for E 0 includes filler odd lines 
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a1, a3, a5, a(k). Likewise, the original field O 0 includes original odd lines b1, b3, 
b5, ... , b(k), and the even filler field that the circuit 50 will generate for O 0 includes 

filler even lines bO, b2, b4 b(k-1). Although the filler lines are shown for clarity in 

explaining the filler-field generation process discussed below, one should understand 
5 that the field buffer 52 does not receive the filler lines. Furthermore, in this 
embodiment, a video frame has an even number of lines. Therefore, because a 
video field has half the lines of the video frame, each of the original video fields E 0 - 
E 3 and Oo - O3 also has an even number of lines. Thus, k-1 is an even number and 
k is an odd number because the first line number is 0. As discussed below, the 
1 0 circuit 50 generates the values of the filler pixels from the original pixels in the 
complimentary field and in other fields of the same polarity. For example, in one 
embodiment, the circuit 50 generates the values of the filler pixels in the filler line a1 
of E 0 from the values of the original pixels in the original lines aO and a2 of E 0 and 
the original lines cO and c2 of E-|. 
1 5 Referring to Figure 7, the generation of motion values for filler pixels in odd 

filler fields is discussed according to an embodiment of the invention. For example 
purposes, the generation of motion values is discussed in conjunction with the 
original even fields E 0 and E1 being represented in a Y, C B , and C R color space and 
having been compressed according to the MPEG 4:2:0 format, it being understood 
20 that the same principles apply to the other original even fields of the Figure 6 
sequence. The generation of motion values for filler pixels in even filler fields is 
discussed below in conjunction with Figure 1 1 . 

Figure 7 is a diagram of the original and filler pixels that compose the original 
and filler lines, respectively, of the even fields E 0 and El The pixels of the original 
25 even field E 0 and its corresponding odd filler field are denoted as l?kx, and the pixels 
of E1 and its corresponding odd filler field are denoted as P'kx, where k denotes the 
line and x denotes the column. Like the original pixels, the filler pixels are arranged 
in 2 x 2 blocks of four pixels (see Figure 2A). For example, a block 80 includes filler 
pixels P 12 , P13, P32, and P 33 , which compose the complimentary filler field for E 0 . 
30 Still referring to Figure 7, the image processing circuit 50 of Figure 4 

generates a respective raw motion value RM for each interior block of filler pixels 
that compose the complimentary filler field for E 0 . (The motion analysis of the 
exterior blocks, i.e., the blocks that include the first two and last two pixels of a line, 
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are discussed below.) The circuit 50 calculates RM from the differences in the 
luminance and chrominance values of the original pixels that border the filler block in 
E 0 and the corresponding original pixels in El For example, the luminance 
difference values for the filler-pixel block 80 are given by the following equation: 

5 

1 ) DY v =\ Y v - Y 'v I; = 0,2; j = 1,2,3,4 

Thus, in this embodiment, there are eight luminance difference values DY: |Y 0 i - 
Y'oil, |Y 02 - Y' 02 |, |Y 03 - Y'osl, |Y 04 - YU |Y 2 i - Y' 21 |, |Y 22 - Y' 22 |, |Y 23 - Y' 23 |, and |Y 24 - 
10 Y' 2 4|. Here, Y 0 i is the luminance value for the original pixel P 0 i, Y' C n is the luminance 
value for P' 0 i, Y 02 is luminance value for the original pixel P 0 2, and so on. The C R 
difference value is given by the following equation: 



15 



2) 



nc = r -c 



where C RO i is the C R value for the block of original pixels in E 0 including P 0 2, P03, P22, 
and P 23 , and C' RO i is the C R value for the block of original pixels in E1 including P' 02 , 
P'03, P'22, and P'23. Similarly, the C B difference value is given by the following 
20 equation: 



3) 



dc = \c -c 



where the blocks of original pixels for C B oi and C' B oi are the same as the blocks for 
25 Croi and C'roi , respectively. 

Still referring to Figure 7, the image processing circuit 50 of Figure 5 
calculates the raw motion value RM 0 i for the block 80 as the maximum of the 
average luminance and chrominance differences according to the following equation: 



30 



4) 



RM 0] = Max 



Vo I YDY„,DC m ,DC 

/ O / =0 j=0 

even J 
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Referring to Figure 8, in one embodiment, the image processing circuit 50 of 
Figure 4 filters the raw motion values RM to reduce the occurrence of false motion 
detection caused by noise and to limit the motion values to four bits, i.e., a maximum 
5 value of 15. Figure 8 is a plot of the filtering algorithm according to an embodiment 
of the invention. Thus, the circuit 50 calculates the filtered motion values FM 
according to the following equation: 
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5) 



„ 1# „ FM = Q 
8<i?M<38 FM = m ~ % 



m> ~™ FM = 15 2 



According to equation (5), the circuit 50 considers a raw motion value RM that 
is less than or equal to 8 to be noise, not true motion, and thus generates a 
corresponding filtered motion value FM = 0. Likewise, the circuit 50 limits the 
maximum FM by clipping to 15 all RM values that are greater than or equal to 38. 

15 Referring back to Figure 7, because the image processing circuit 50 of Figure 

5 derives the luminance difference values DY from groups of four original pixels in 
the same original line, the circuit 50 cannot use the above-described technique to 
generate raw motion values for the filler pixels at the beginnings and ends of filler 
lines. For example, the circuit 50 uses four pixels P 0 i, Po2, P03, and P 0 4 in the same 

20 line to generate some of the DY values for the filler block 80. Therefore, P 0 i 

precedes the block 80 in a horizontal direction, and P 04 proceeds the block 80 in the 
horizontal direction. But referring to the filler blocks 82 and 84, because no pixels 
precede the block 82 and no pixels proceed the block 84, equation (1) is invalid for 
these blocks. Thus, in one embodiment, the circuit 50 calculates no raw or filtered 

25 motion values for the filler blocks 82 and 84 and the other filler blocks that contain 
the first two or last two filler pixels of respective filler lines. Alternatively, the circuit 
50 assigns predetermined filtered motion values to these blocks. For example, in 
one embodiment, the circuit 50 assigns these blocks a constant FM value or the 
same FM value as calculated for the adjacent block in the same filler line. For 

30 example, according to the latter approach, the circuit 50 sets the filtered motion 
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value FMoo for the block 82 equal to FM01, which the circuit 50 calculates for the 
adjacent block 80 as described above. 

Figure 9 illustrates the content layout of a motion-value buffer 90 for storing 
filtered motion values FM for filler fields derived from MPEG 4:2:0 original fields 

5 according to an embodiment of the invention. Referring to Figure 4, in one 

embodiment, the image processing circuit 50 dedicates a portion of the memory 58 
as the buffer 90, although the buffer 90 may reside in another memory. The circuit 
50 includes only one buffer 90, and updates the contents of this buffer for each filler 
field. A procedure for updating the buffer 90 is discussed below in conjunction with 

10 Figure 16. 

The storage locations FM of the buffer 90 respectively correspond to the filler- 
pixel blocks described in conjunction with Figure 7. For example, the location FM01 
stores the filtered motion value FM 0 i, which corresponds to the block 80 of Figure 7. 
Furthermore, if the image processing circuit 50 of Figure 4 assigns motion values to 

15 the beginning-line and ending-line pixel blocks, then the buffer 90 also includes 
optional locations that are shown in dashed line. For example, the optional location 
FMoo corresponds to the beginning-line block 82 of Figure 7, and the optional 
location FM 0 ( X /2) corresponds to the ending-line pixel block 84. 

Referring to Figures 7 and 9, because the dimensions of the filler-pixel blocks 

20 such as the block 80 are 2 x 2 and because the image processing circuit 50 of Figure 
4 calculates one FM value per block, the horizontal dimension of the buffer 90 is 
either half or two pixels less than half the horizontal dimension of the original and 
filler fields. Specifically, if the motion-value buffer 90 includes the optional storage 
locations shown in dashed line, then the horizontal dimension of the buffer 90 is half 

25 the horizontal dimension of the original and filler fields. For example, if the original 
and filler fields have horizontal dimensions of x = 720 pixels, then the buffer 90 is 
720 -r 2 = 360 memory locations wide. Alternatively, if the motion-value buffer 90 
does not include the optional storage locations shown in dashed line, then the 
horizontal dimension of the buffer 90 is half the horizontal dimension of the original 

30 and filler fields minus two pixels. For example, if the original and filler fields have 
horizontal dimensions of x = 720 pixels, then the buffer 90 is (720 -h 2) - 2 = 358 
memory locations wide. 
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Similarly, the vertical dimension of the buffer 90 is one-half the vertical 
dimension of the original and filler fields, and thus one-fourth the vertical dimension 
of the resulting progressive frames generated by the image processing circuit 50 of 
Figure 4. This is true whether or not the buffer 90 includes the optional storage 
5 locations. For example, if the original and filler fields each have vertical dimensions 
of k/2 = 240 lines — the corresponding progressive frames have k = 2 x 240 = 480 
lines — then the buffer 90 is k/4 = 240 + 2 — 480 -s- 4 with respect to the 
corresponding progressive frames — = 120 memory locations high. 

Figure 10 illustrates the content layout of a motion-trace buffer 92 for storing 

1 0 motion-trace values MT for filler fields derived from MPEG 4:2:0 original fields 
according to an embodiment of the invention. As discussed below in conjunction 
with Figure 16, each motion-trace value specifies the number of filler fields for which 
a respective FM value is valid. Referring to Figure 4, in one embodiment, the image 
processing circuit 50 dedicates a portion of the memory 58 as the buffer 92, although 

1 5 the buffer 92 may reside in another memory. The circuit 50 includes only one buffer 
92, and updates the contents of this buffer for each filler field. A procedure for 
updating the buffer 92 is discussed below in conjunction with Figure 16. 

The storage locations MT of the buffer 92 respectively correspond to the filler- 
pixel blocks described in conjunction with Figure 7. For example, the location MT 0 i 

20 stores the motion-trace value MT 0 i, which corresponds to the block 80 of Figure 7, 
and thus which corresponds to the location FM 0 i of the motion-value buffer 90 of 
Figure 9. Furthermore, if the image processing circuit 50 of Figure 4 assigns motion 
values, and thus motion-trace values, to the beginning-line and ending-line pixel 
blocks, then the buffer 92 also includes optional locations that are shown in dashed 

25 line. For example, the optional location MT 0 o corresponds to the beginning-line block 
82 of Figure 7, and thus corresponds to the location FM 0 o of the motion-value buffer 
90. Similarly, the optional location MT 0(x /2) corresponds to the ending-line pixel block 
84 of Figure 7, and thus corresponds to the location FM 0 (x/2) of the motion-value 
buffer 90. 

30 Still referring to Figure 10, the motion-trace buffer 92 has the same 

dimensions as the motion-value buffer 90 as discussed above in conjunction with 
Figure 9. Furthermore, in one embodiment, each storage location MT is four bits 
wide. 
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Referring to Figures 9 and 10, the motion-value and motion -trace buffers 90 
and 92 are significantly smaller than the motion memories of many prior image 
processing circuits. Furthermore, one can vary a motion-trace value within a 
predetermined range to vary the number of filler fields for which a respective FM 
5 value is valid without increasing the size of the buffer 92. 

Referring to Figures 7, 9, and 10, as long as k + 1 is divisible by four, then 
there are no partial (2x1) filler-pixel blocks at the bottom of the complimentary odd 
filler fields for the even fields E. For example, the last row of filler blocks include 
respective pixels from the filler lines a(k-2) and a(k), and there are no unpaired filler 

10 lines below this. Conversely, if k + 1 is not divisible by four, then there is a row of 
partial filler blocks that include pixels from only one filler line a(k). In this situation, 
the image processing circuit 50 of Figure 5 can calculate the raw and filtered motion 
values and the motion-trace values for these partial filler blocks in a number of ways. 
For example, the circuit 50 can set the raw and filtered motion values and the 

1 5 motion-trace value for a partial filler block equal to the raw and filtered motion values 
and motion-trace value, respectively, for the full filler block immediately above the 
partial filler block. For example, referring to Figures 7 and 9, if the filtered-motion- 
value location FM (kM)1 corresponds to a partial filler block in the filler field that 
compliments E 0 , then the circuit 50 can set FM (k / 4 )i = FM {m -w and MT^i = MT ((k / 4) . 

20 

Referring to Figure 1 1 , the generation of motion values for filler pixels in even 
filler fields is discussed according to an embodiment of the invention. For example 
purposes, the generation of motion values is discussed in conjunction with the 
original odd fields O 0 and Oi being represented in a Y, C B , and C R color space and 

25 having been compressed according to the MPEG 4:2:0 format, it being understood 
that the same principles apply to the other original odd fields of the Figure 6 
sequence. The generation of motion values for filler pixels in odd filler fields is 
discussed above in conjunction with Figure 7. 

Figure 11 is a diagram of the original and filler pixels that compose the original 

30 and filler lines, respectively, of the odd fields O 0 and Cv The pixels of the original 
odd field Oo and its corresponding even filler field are denoted as Pkx, and the pixels 
of Oi and its corresponding even filler field are denoted as P'kx, where k denotes the 
line and x denotes the column. Like the original pixels, the filler pixels are arranged 
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in 2 x 2 blocks of four pixels (see Figure 2A). For example, a block 94 includes filler 
pixels P 0 2, P03, P22, and P23, which compose the complimentary filler field for O 0 . 

Stiil referring to Figure 11, the calculation of the difference values DY, DC R , 
and DC B and the raw and filtered motion values RM and FM for the even filler fields 
are similar to the respective DY, DC R , DC B , RM, and FM calculations for the odd filler 
fields as described above in conjunction with Figure 7. For example, DY, DC R , DC B , 
and RM for the block 94 are given by the following equations: 



6) DY =7.-7' 
) V-j '^ = 1,3;;= 1,2,3,4 



7) DC =\C -C , 



8) DC B0 \ — \C BOl C BOl \ 



15 9) RM 0l =Max 



Y % Z iDY u ,DC ROl ,DC BOl 



FM01 is given by equation (5). 

Referring to Figures 7, 9, and 11 and equations (5) and (9), the location FM01 
of the motion-value buffer 90 corresponds to the block 80 of Figure 7 and to the 

20 block 94 of Figure 1 1 . Therefore, the image processing circuit 50 of Figure 4 stores 
only one FM01 value — FM01 for the block 80 or FM oi for the block 94 — in the 
location FM 0 i. The procedure for selecting which FM01 to store is discussed below in 
conjunction with Figure 16. 

Furthermore, as discussed above in conjunction with Figure 7, the image 

25 processing circuit 50 of Figure 4 cannot use the above-described technique to 

generate raw motion values for the filler blocks such as blocks 96 and 98 that include 
filler pixels at the beginnings and ends of filler lines. Thus, in one embodiment, the 
circuit 50 calculates no raw or filtered motion values for the filler blocks 96 and 84 
and the other filler blocks containing the first two or last two filler pixels of respective 

30 filler lines. Alternatively, the circuit 50 assigns predetermined filtered motion values 
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to these blocks. For example, in one embodiment, the circuit 50 assigns these 
blocks a constant FM value or the same FM value as calculated for the adjacent 
block in the same filler line. For example, according to the latter approach, the circuit 
50 sets the filtered motion value FM 0 o for the block 96 equal to FM 0 i, which the 
circuit 50 calculates for the adjacent block 94 as described above. 

Referring to Figures 9, 10, and 11, as discussed above in conjunction with 
Figure 7, as long as k + 1 is divisible by four, then there are no partial (2 x 1) filler- 
pixel blocks at the bottom of the complimentary even filler fields for the odd fields 0. 
Conversely, if k + 1 is not divisible by four, then there is a row of partial filler blocks 
that include pixels from only one odd filler line b(k-1). In this situation, the image 
processing circuit 50 of Figure 5 can calculate the raw and filtered motion values and 
the motion-trace values for these partial filler blocks in a number of ways as 
discussed above in conjunction with the filler field of Figure 7. 

Referring to Figure 12, the generation of motion values for filler pixels in odd 
filler fields is discussed according to another embodiment of the invention in which 
the original even fields of Figure 6 are represented in a Y, Cb, and C R color space 
and have been compressed and decompressed according to the MPEG 4:2:2 
format Like the original pixels, the filler pixels are arranged in 2 x 1 blocks of two 
pixels (see Figure 3A). For example, a block 98 includes filler pixels P 12 and Pi 3 , 
and a block 100 includes filler pixels P 32 and P33. Thus, the major difference 
between Figures 7 and 12 is that in Figure 12, the filler-pixel blocks contain two 
pixels instead of four pixels. Therefore, DY, DC R , DC B , and RM for the block 98 are 
given by the following equations: 

1 0) DYy =|^y -Vy |; = ; - = 4 

1 1 ) DC Ri\ =\ C M "C'&l |/ = 0,2 

1 2) DC Ba =\C m -C Bi[ | z - = 0(2 
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13) RM Q , = Max 



U^' 1 /Z ;-„„=0 /Z U„=° 
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FM01 is given by equation (5). Furthermore, the calculation of DY is the same as for 
the 4:2:0 format, and thus equation (10) is the same as equation (5). Furthermore, 
because the 4:2:2 format calls for one C R and one C B value for each 2 x 1 block of 
original pixels, the calculation of DC R includes taking the difference between C RO i, 
which corresponds to pixels P 02 and P 03 of E 0 , and C'roi, which corresponds to P' 02 
and P' 03 of El and taking the difference between Cru, which corresponds to pixels 
P22 and P23 of E 0 , and C' R11 , which corresponds to pixels P'22 and P' 23 of El A 
similar analysis applies to DC B . 

Still referring to Figure 12, the image processor circuit 50 of Figure 5 
calculates difference and raw motion values for the 2 x 1 block 100 according to the 
following equations: 
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14) DY =\Y -Y' I 

v ' 9 ,J \i = 2A;j = 1,2,3,4 



15) DC Rn =\C Rl ,-C 



ftl| / = 2,4 



20 



16) DC Bll =\C m -C BllU _ 



i = 2,4 



17) RM n = Max 



y % i iDY,,y ^c^y i DCm 

< em) -^-l , 2 , =2 
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Furthermore, for the same reasons discussed above in conjunction with 
Figure 7, the image processing circuit 50 of Figure 4 cannot use the above-described 
technique to generate raw or filtered motion values for the filler-pixel blocks 102 and 
104 and other filler blocks containing filler pixels at the beginnings and ends of filler 
lines. Thus, the circuit 50 generates filtered motion values for these filler blocks as 
discussed above in conjunction with Figure 7. 
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In addition, if k +1 is divisible by two, the last filler line a(k) of the filler field that 
compliments E 0 is not "sandwiched" between two original lines of E 0 . Therefore, the 
circuit 50 calculates DY, DC R , DC B , and RM for a last-line filler block such as the 
block 106 using original pixels in only the last lines a(k-1) and c(k-1), respectively, of 
the original fields E 0 and El For example, the circuit 50 calculates the difference 
and raw motion values for the pixel block 106 according to the following equations: 



1 8) DYp-vj -\Y (k -\)j ~ Y \k-i)j \j _ 1 23 4 



10 19) DC^^C^-C^ 



20) DC B ( k _ X)l - Ic^^, -C'^.jJ 



21) RM^_ m =Max 



l/ 4 



Thus, for example DY for the block 106 is calculated using the luminance values for 
the pixels P^, P (k . 1)2) p (k . 1)3i and P (k . 1)4 from E 0 and P'^do, PVdl p, (k-i )2l PVds, 
and P' (k . 1)4 from El Likewise, DC R and DC B are calculated from the C R and C B 
values, respectively, for the 2 x 1 blocks of original pixels that include P (k . 1)2 and P (k . 
20 1)3 from E 0 and P' (k . 1)2 and P' (k . 1)3 from El The circuit 50 uses equation (5) to 

calculate the filtered motion values including filter equations 5 are used to calculated 
the filtered motion values. 

Figure 13 illustrates the content layout of a motion-value buffer 108 for storing 
filtered motion values FM for filler fields derived from MPEG 4:2:2 original fields 

25 according to an embodiment of the invention. Referring to Figure 4, in one 

embodiment, the image processing circuit 50 dedicates a portion of the memory 58 
as the buffer 108, although the buffer 108 may reside in another mesmory. The circuit 
50 includes only one buffer 108, and updates the contents of this buffer for each filler 
field. A procedure for updating the buffer 108 is discussed below in conjunction with 

30 Figure 16. 
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Referring to Figures 12 and 13, because the dimensions of the filler-pixel 
blocks such as the block 98 are 2 x 1 and because the image processing circuit 50 of 
Figure 4 calculates one FM value per block, the horizontal dimension of the buffer 
108 is either half or two pixels less than half the horizontal dimension of the original 
5 and filler fields. Specifically, if the motion-value buffer 1 08 includes the optional 
storage locations shown in dashed line, then the horizontal dimension of the buffer 
108 is half the horizontal dimension of the original and filler fields. For example, if 
the original and filler fields have horizontal dimensions of x = 720 pixels, then the 
buffer 108 is X/2 = 720 + 2 = 360 memory locations wide. Alternatively, if the 

1 0 motion-value buffer 1 08 does not include the optional storage locations shown in 
dashed line, then the horizontal dimension of the buffer 108 is half the horizontal 
dimension of the original and filler fields minus two pixels. For example, if the 
original and filler fields have horizontal dimensions of x = 720 pixels, then the buffer 
90 is (x/2) - 2 = (720 + 2) - 2 = 358 memory locations wide. 

1 5 Similarly, the vertical dimension of the buffer 1 08 is the same as the vertical 

dimension of the original and filler fields, and thus one-half the vertical dimension k 
of the resulting progressive frames generated by the image processing circuit 50 of 
Figure 4. This is true whether or not the buffer 108 includes the optional storage 
locations. For example, if the original and filler fields each have vertical dimensions 

20 of k/2 = 240 lines — the corresponding progressive frames have k = 2 x 240 = 480 
lines — then the buffer 108 is k/2 = 240 — 480 ^ 2 with respect to the corresponding 
progressive frames — = 240 memory locations high. 

Thus, the motion-value buffer 108 has the same horizontal dimension and 
twice the vertical dimension as the motion-value buffer 90 of Figure 9. 

25 Figure 14 illustrates the content layout of a motion-trace buffer 110, which is 

similar to the buffer 92 of Figure 10 except that it stores motion-trace values MT for 
filler fields derived from MPEG 4:2:2 original fields according to an embodiment of 
the invention. The storage locations MT of the buffer 110 respectively correspond to 
the filler-pixel blocks described in conjunction with Figure 12. For example, the 

30 location MT 0 i stores the motion-trace value MT 0 i, which corresponds to the block 98 
of Figure 12, and thus which corresponds to the location FM 0 i of the motion-value 
buffer 108 of Figure 13. Furthermore, if the image processing circuit 50 of Figure 4 
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assigns motion values, and thus motion-trace values, to the beginning-line and 
ending-line pixel blocks, then the buffer 108 also includes optional locations that are 
shown in dashed line. For example, the optional location MT 00 corresponds to the 
beginning-line block 102 of Figure 12, and thus corresponds to the location FM 00 of 
the motion-value buffer 108. 

Still referring to Figure 14, the motion-trace buffer 1 10 has the same 
dimensions as the motion-value buffer 108 as discussed above in conjunction with 
Figure 13. 

Referring to Figure 15, the generation of motion values for filler pixels in even 
filler fields is discussed for the original odd fields of Figure 6 being represented in a 
Y, C B , and C R color space and having been compressed and decompressed 
according to the MPEG 4:2:2 format. The calculation of the difference values DY, 
DC Rl and DC B and the raw and filtered motion values RM and FM for the even filler 
fields are similar to the respective DY, DC Rl DCs, RM, and FM calculations for the 
1 5 odd filler fields as described above in conjunction with Figure 12. For example, DY, 
DC R , DC b , and RM for the block 1 12 are given by the following equations: 



22) DY =\Y -T I 

' IJ I u 'J \i = 1,3; j = 1,2,3,4 
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20 23) DC m =\C Rll -C m 



25 



i = 1,3 



24) DC Bll =\C m -C Bll , 



i = U 



25) RM U = Max 



V % i iDY v ,y j]Dc m ,y tDc Bn 



FM 0 i is given by equation (5). Furthermore, for the same reasons discussed 
above in conjunction with Figure 7, the image processing circuit 50 of Figure 4 
cannot use the above-described technique to generate raw or filtered motion values 
for the filler-pixel blocks 1 14 and 1 16 and other filler blocks containing filler pixels at 
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the beginnings and ends of filler lines. Thus, the circuit 50 generates filtered motion 
values for these filler blocks as discussed above in conjunction with Figure 7. 

In addition, because the first filler line bO of the filler field that compliments O 0 
is not "sandwiched" between two original lines of O 0 , the circuit 50 calculates DY, 
5 DC R , DC b , and RM for a first-line filler block such as the block 118 using original 
pixels in only the second lines b1 and d1, respectively, of the original fields O 0 and 
0<\. For example, the circuit 50 calculates the difference and raw motion values for 
the pixel block 1 18 according to the following equations: 

10 26) DY X] =\Y xr r Xj \ j = lxXA 

27) DC R01 =\C R01 -C RQl \ 

28) ix: B01 =|c sol -c B01 | 
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29) RM 0l =Max 



1/ 

A Y.DY Xj ,DC ROl ,DC BQ1 



FM01 is given by equation (5). Similarly, if k + 1 is not divisible by two, the last filler 

line bk of the filler field is not "sandwiched" between two original lines of O 0 . 
20 Therefore, the circuit 50 calculates DY, DC R , DC B , and RM for a last-line filler block 

using original pixels in only the lines b(k-1) and d(k-1), respectively. 

Referring to Figures 12, 13, and 15, the location FM 0 i of the motion-value 

buffer 108 corresponds to the block 98 of Figure 12 and to the block 108 of Figure 

15. Therefore, the image processing circuit 50 of Figure 4 stores only one FM 0 i 
25 value — FM 0 i for the block 98 or FM 0 i for the block 1 08 — in the location FM 0 i . The 

procedure for selecting which FM 0 i to store is discussed below in conjunction with 

Figure 16. 

Figure 16 is a flow diagram of the technique that the image processing circuit 
50 of Figure 4 implements to initialize and update the motion-value buffer 90 and 
30 motion-trace buffer 92 of Figures 9 and 1 0 (4:2:0 format) or the buffers 1 08 and 1 1 0 
of Figures 13 and 14 (4:2:2 format) according to an embodiment of the invention. 
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For clarity, this technique is discussed in conjunction with the buffers 90 and 92, it 
being understood that the technique is similar for the buffers 108 and 110. 

Referring to Figure 5 and block 1 12 of Figure 16, the processor 56 loads 0 
into all storage locations of the buffers 90 and 92. 
5 Referring to block 1 14, the processor 56 then retrieves the next filtered motion 

value FMkx, which it is previously calculated and stored in the memory 58. 

Referring to blocks 1 16 and 1 18, if FM^ is greater than or equal to the current 
contents of the location k, x of the motion-value buffer 90, then the processor 56 
overwrites the location k, x with FM^. Next, referring to block 120, the processor 56 

1 0 loads an initial MT value into the k, x location of the motion-trace buffer 92. In one 
embodiment, the initial MT value equals 5. 

Conversely, referring to blocks 116 and 122, if FMkx is less than the current 
contents of the location k, x location of the motion-value buffer 90, then the 
processor 56 analyzes the contents of the k, x, location of the trace buffer 92. If the 

1 5 contents equals 0, then, referring to block 124, the processor 56 loads 0 into the k, x 
location of the motion-value buffer 90 to indicate that there is no motion associated 
with the respective filler-pixel block of the current filler field. Conversely, referring to 
block 126, if the contents of the k, x location of the trace buffer 92 does not equal 0, 
then the processor 56 decrements the contents by a predetermined value D. In one 

20 embodiment, D = 1. 

Referring to block 128, the processor 56 processes the next FM^ value in a 
similar manner. 

Therefore, by varying the values of D and the initial MT, one can vary the 
maximum number of filler fields that a motion value will influence. For example, 

25 referring to Figures 6, 7, 9, 10, 1 1 , and 16, suppose that the initial MT = 5, D = 1 , 
FM01 = 1 5 for the filler-pixel block 80 of the filler line a1 for E 0 , and FM 0 i < 1 5 for the 
corresponding filler-pixel blocks of the filler lines b0-b2, c1-c3, d0-d2, e1-e3, and fO- 
f2 for O 0 , Ei, Oi, E 2 , and 0 2 , respectively. Thus, according to the flow diagram of 
Figure 16, for the block 80, the processor 56 loads FM 0 i = 15 into the FM 0 i location 

30 of the motion-value buffer 90 and loads 5 into the MT 0 i location of the trace buffer 
92. Next, because FM 0i < 15 for the filler-pixel block 94 (Figure 11), the processor 
56 leaves the previous FM 0 i = 15 in the FM 0 i location of the buffer 90 and 
decrements the contents of the MT 01 location of the buffer 92 to 4. The processor 56 
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processes FM01 < 15 for the filler-pixel blocks (not shown) for c1-c3, d0-d2, e1-e3, 
and f0-f2 in a similar manner. After processing the filler-pixel block of f0-f2, however, 
the location MT 0 i of the trace buffer 92 equals 0. Thus, as discussed below, even if 
the processor 56 detects no subsequent motion, the motion detected between E 0 
5 and Ei influences the values of the filler pixels in the filler fields complimentary to six 
consecutive original fields: E 0 , O 0 , O,, E 2 , and 0 2 . Thus, unlike many of the prior 
image processing circuits, the image processing circuit 50 allows detected motion to 
influence the filler-pixel values in more than four filler fields. Furthermore, one can 
vary the initial MT value or D to vary the number of motion-affected filler fields 

1 0 without increasing the sizes the buffers 90 and 92. 

Figure 17 is a diagram of the original pixels that the image processing circuit 
50 of Figure 4 uses to calculate direction values and to spatially interpolate a pixel 
value for a filler pixel according to an embodiment of the invention. For example 
purposes, the filler pixel is P 33 of the filler line a3 of Figure 7, it being understood that 

1 5 the following discussion applies to other filler pixels except the first and last pixels of 
each filler line. Calculating values for these filler pixels is discussed below. As 
discussed below in conjunction with Figures 18A - 18E, the circuit 50 calculates the 
direction values and spatially interpolates the pixel value for P 33 from the three 
original pixels P 22 , P 2 3, and P 24 above P 33 and three pixels P 42 , P 43 , and P44 below 

20 P 33 . 

Referring to Figures 18A - 18E, in one embodiment of the invention, the 
image processing circuit 50 recognizes three edge directions and two thin-line 
directions with respect to the pixel diagram of Figure 17. In determining the direction 
values, the circuit 50 uses only the luminance values Y of the original pixels in the 
25 pixel diagram. 

Referring to Figure 18A, the circuit 50 recognizes a 45° - 225° edge if the 
original pixels P 24 and P 42 have similar Y values. For example, if the normal to the 
edge points toward the lower right of the pixel group, then the pixels P 22 and P 23 
have Y values similar to the Y values of P 42 and P 24 . Conversely, if the normal to the 
30 edge points toward the upper left of the pixel group, then the pixels P 43 and P 44 have 
Y values similar to the Y values of P 24 and P 42 . 

Referring to Figure 18B, the circuit 50 recognizes a thin line that extends 
through the pixels P 23 , P 2 4, P42, and P 43 at approximately 63° - 243°. This thin line is 
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characterized by P 22 and P M having Y values that are significantly different than the 

Y values of P 23 , P 24 , P 42 , and P 43 . - 

Referring to Figure 18C, the circuit 50 recognizes a 90° - 270° edge if the 
original pixels P 23 and P 43 have similar Y values. For example, if the normal to the 

5 edge points toward the right of the pixel group, then the pixels P 22 and P 42 have Y 
values similar to the Y values of P 23 and P 43 . Conversely, if the normal to the edge 
points toward the left of the pixel group, then the pixels P 24 and P 44 have Y values 
similar to the Y values of P 23 and P 43 . The circuit 50 also recognizes a 90° - 270° 
edge if all of the pixels P 22 , P 23 , P 24 , P 42 , p 43 , and P 44 have similar Y values. 

0 Referring to Figure 1 8D, the circuit 50 recognizes a thin line that extends 

through the pixels P 22 , P 23l P 43 , and P 44 at approximately 1 17° - 297°. This thin line 
is characterized by P 24 and P 42 having Y values that are significantly different than 
the Y values of P 22 , P 23 , P43, and P 44 . 

Referring to Figure 18E, the circuit 50 recognizes a 135° - 315° edge if the 

5 original pixels P 22 and P M have similar Y values. For example, if the normal to the 
edge points toward the lower left of the pixel group, then the pixels P 23 and P 24 have 

Y values similar to the Y values of and P^. Conversely, if the normal to the edge 
points toward the upper right of the pixel group, then the pixels P 42 and P 43 have Y 
values similar to the Y values of P 22 and P44. 

0 The direction values DV are calculated according to column 2 of Table I. 



Table I 



Directions 


Pixel difference along directions 


If minimum of pixel 
differences is below T e d ge 
threshold, estimate missing 
pixel value P s as: 


45°-225° 
(Fig. 18A) 


DV 45 . 225 = |P 24 - P 42 | + offset 


Ps33=(P24 + P 42 )/2 


63°-243° 
(Fig. 18B) 


DV 63 . 243 = (|P 24 - P 43 | + |P 23 - P 42 |)/2 


Ps33 = (P 24 + P 42 + P 23 + P 43 )/4 


90°-270° 
(Fig. 18C) 


DVgo-270 = I P 23 - P 43 i 


Ps33 = ( P 23 + P 43 ) / 2 
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117°-297° 
(Fig. 18D) 


DV 117 .297=(| P23 - P44 |+| P22 - P43 
|)/2 

1/ 


Ps3 = (P22 + P44 + P23 + P 43 )/4 


135°-315° 
(Fig. 18E) 


DV 135 -315 = | P22 - P44 | + offset 


Ps = ( P22 + P44 ) / 2 


No dominant edge direction if min. of above values 

> Tedge 


Minimum DV > <^n 
estimate P s as: 
P S = (P 23 + P 43 )/2 



The circuit 50 calculates all the DV values according to column 2 of Table I, 
identifies the minimum DV value, and compares the minimum DV to a threshold 
Tedg e . If the minimum DV is greater than T ed ge, then processor 56 identifies by default 
5 a 90°-270° edge (Figure 1 8C), and thus spatially interpolates Y, C R) and C B values 
(collectively represented as P s in Table I) for P 33 equal to the average of the 
respective Y, C R , and C B values of P 23 and P 43 as shown in the last row of column 3 
of Table I. Conversely, if the minimum DV is less than or equal to Tedge, then the 
processor 56 calculates P s according to the equation corresponding to the minimum 

1 0 DV. For example, if DV45-225 is the minimum, then the processor 56 calculates P s 
equal to the average of the values of P 24 and P 42 . If DV 63 -243 is the minimum value, 
then the processor 56 calculates P s equal to the average of the values of P 23 , P 24 , 
P42, and P 43 . If DV90-270 is the minimum value, then the processor 56 calculates P s 
equal to the average of the values of P 23 and P 43 . If DV 117 - 2 g 7 is the minimum value, 

1 5 then the processor 56 calculates P s equal to the average of the values of P 22 , P 23 , 
P 43 , and P44. And if DV 135 . 315 is the minimum value, then the processor 56 calculates 
P s equal to the average of the values of P 22 and P44. 

Tedge is an empirically determined constant. In one embodiment, it is in the 
range of approximately 40 - 50. 

20 The inventor has determined that the 63°-243° and 1 1 7°-297° thin lines have 

a tendency to be misinterpolated as 45°-225° and 135°-315° edges. Therefore, the 
circuit 50 adds an offset to DV 135 -3i 5 and DV 45 -225to effectively offset this 
misinterpolation by favoring the detection of thin lines. In one embodiment, the 
processor 56 calculates the offset according to the following equation: 
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30) offset = 



1 0 ~ 20 when\P 22 - P 24 1 > t Vme 

and\P 42 -P H \>t line 
0 Otherwise 



where t| ine is a threshold empirically determined to be approximately 30 in one 
embodiment of the invention. 
5 Referring to Figures 7, 1 1 , and 17, the processor 56 also temporally 

interpolates Y, C R , and C B (collectively P t ) for each filler pixel from the corresponding 
original pixels in an adjacent original field having the same polarity as the filler field. 
For example, the processor 56 calculates P t for the filler pixel P 33 in the filler line a3 
of E 0 (Figure 7) equal to the luminance and chromanance values of the original pixel 
1 0 P 33 in the original line b3 of O 0 (Figure 1 1 ). 

Next, the processor 56 calculates the final values P f of the filler pixels 
according to the following equations: 

31 ) a = Max(FM^ I _ l)j , FM ip FM( M)j ) 

15 

32) P f= 2. iaPs + {l . a)Pi) 

Specifically, the processor 56 calculates P f equal to the sum of a-weighted P s 
and (1 - a) weighted P t . a equals the maximum of the FM value of the filler pixel for 

20 which P f is being calculated, and the FM values of the filler-pixel blocks above and 
below the filler pixel. For example, referring to Figures 7 and 9, a for the filler pixel 
P33 is the maximum of FM 0 i and FMn (there is no filler pixel block above the block 
80, and thus no FM location of the buffer 90 above FM 0 i.) Taking the maximum of 
the three closest FM values in a vertical direction ensures that the spatially 

25 interpolated value P s is given the greatest weight where there is detected motion. 
Furthermore, as stated above in conjunction with Figure 8, the maximum value of 
FM, and thus the maximum value of a, is 15. Therefore, left side of equation (32) is 
divided by 15 for normalization. 

Referring to equation (32), if there is significant motion such that a is relatively 

30 large, then P f is weighted heavily toward the spatially interpolated value P s . 
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Conversely, if there is little motion such that a is relatively small, then P f is weighted 
heavily toward the temporally interpolated value P t . 

Referring to Figure 7, the generation of values for filler pixels at the 
beginnings and ends of filler pixel lines is discussed. For example purposes, the 
5 filler pixel P 10 of E 0 is discussed, it being understood that the following discussion 
applies to Pn and other filler pixels at the beginning and ending of filler lines. 

If the image processing circuit 50 assigns a predetermined value FM 0 o to P 10 a 
and the other beginning- and end-of-line filler pixels as described above, then the 
processor 56 calculates P s , P t , and P f for P 10 as discussed above. 
1 0 Conversely, if the processing circuit 50 does not assign a predetermined 

. value FMoo to P 10 and the other beginning- and end-of-line filler pixels as described 
above, then the processor 56 calculates P f a manner other than that described 
above. For example, the processor 56 may exclusively spatially interpolate P f for P 10 
equal to the average of the respective Y, C R , and C B values of the vertically adjacent 
15 original pixels P 00 and P 20 . Or, the processor 56 may exclusively temporally 

interpolate P f for P10 equal to the Y, C R , and C B values of the corresponding original 
pixel P 10 in the adjacent original odd field O 0 . 

From the foregoing it will be appreciated that, although specific embodiments 
of the invention have been described herein for purposes of illustra tion, various 
20 modifications may be made without deviating from the spirit and scope of the 
invention. 
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What is claimed: 

1 . An image processing circuit, comprising: 
a processor operable to, 

receive a value of an original pixel of an original first video image and a 
5 value of an original pixel of an original second video image, 

generate a first pixel-value component from the value of the original 
pixel of the original first video image, 

generate a second pixel-value component from the value of the original 
pixel of the original second video image, 
1 0 generate a value of a filler pixel from the first and second pixel-value 

components, and 

combine the filler pixel and the original first video image to generate a 
resulting video image. 

15 2. The image processing circuit of claim 1 wherein the original second 

video image follows the original first video image in a sequence of video images. 

3. The image processing circuit of claim 1 wherein: 
the original first video image comprises an original field; and 
20 the processor is operable to generate the resulting video image by, 

generating a filler field that includes the filler pixel and that is 
complimentary to the original field, and 

merging the original and filler fields. 



25 4. The image processing circuit of claim 1 wherein: 

the original first video image comprises a first original field that includes the 
original pixel of the original first video image; 

the original second video image comprises a second original field that 
includes the original pixel of the original second video image; and 
30 the processor is operable to generate the resulting video image by, 

generating a filler field that includes the filler pixel and that is 
complimentary to the first and second original fields, and 
combining the filler field and the first original field. 
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5. The image processing circuit of claim 1 wherein the processor is 
operable to: 

generate the first pixel-value component equal to the value of the original pixel 
5 of the original first video image; and 

generate the second pixel-value component equal to the value of the original 
pixel of the original second video image, 

6. The image processing circuit of claim 1 wherein the processor is further 
10 operable to: 

weight the first and second pixel-value components; and 
generate the value of the filler pixel from the weighted first and second pixel- 
value components. 

7. The image processing circuit of claim 1 wherein the processor is further 
operable to: 

generate a motion value from the values of the original pixels of the original 
first and second video images; 

generate from the motion value first and second weighting factors; 
generate a weighted first pixel-value component by combining the first 
weighting factor and the first pixel-value component; 

generate a weighted second pixel-value component by combining the second 
weighting factor and the second pixel-value component; and 

generate the value of the filler pixel from the weighted first and second pixel- 
value components. 

8. An image processing circuit, comprising: 
a processor operable to, 

receive a value of an original pixel of a first original video image and a value 
30 of an original pixel of a second original video image that follows the first original 
video image, 

generate a motion value for a first filler video image from the values of 
the original pixels of the first and second original video images, and 

31 
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cause the motion value to indicate motion for a predetermined number 
of filler video images following the first filler video image if the motion value 
indicates motion with respect to the first filler video image. 

5 9. The image processing circuit of claim 8 wherein the processor is 

operable to generate the motion value equal to a difference between the values of 
the original pixels of the first and second original video images. 

10. The image processing circuit of claim 8 wherein the predetermined 
1 0 number equals five. 

1 1 . The image processing circuit of claim 8 wherein: 

the motion value indicates motion if the motion value equals a nonzero 
number; and 

15 the processor is operable to maintain the motion value equal to a nonzero 

number for the predetermined number of filler video images following the first filler 
video image if the motion value equals a nonzero number with respect to the first 
filler video image. 



20 1 2. The image processing circuit of claim 8 wherein the processor is 

operable to generate the motion value by: 

generating a raw motion value for the first filler video image from the values of 
the original pixels of the first and second original video images; and 

filtering the raw motion value to generate the motion value. 

25 

13. The image processing circuit of claim 8 wherein: 

the first original video image comprises a first original video field having a 
polarity; and 

the second original video image comprises a second original video field 
30 having the same polarity as the first original video field. 

14. The image processing circuit of claim 8 wherein the first original video 
image and the original video images following the first original video image compose 
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a sequence of original video images that includes the second original video image 
and a third original video image located between the first and second original video 
images. 



5 1 5. The image processing circuit of claim 8 wherein: 

the first original video image and the original video images following the first 
original video image compose a sequence of original video images that includes the 
second original video image and a third original video image located between the 
first and second original video images; 
"10 the first original video image comprises a first original video field having a 

polarity; 

the second original video image comprises a second original video field 
having the same polarity as the first original video field; and 

the third original video image comprises a third original video field having a 
1 5 polarity opposite to the polarities of the first and second original video fields. 

16. An image processing circuit, comprising: 
a processor operable to, 

receive first and second sets of pixel values for first and second groups, 
20 respectively, of pixels in a video image; 

generate direction values from the first and second sets of pixel values 
for a filler pixel disposed in the video image between the first and second 
groups of pixels; and 

generate a value for the filler pixel based on the direction values. 

25 

17. The image processing circuit of claim 16 wherein: 

the first and second groups of pixels each respectively include three 
horizontally aligned pixels; 

the pixels of the first group are vertically aligned with the respective pixels of 
30 the second group; and 

the filler pixel is vertically aligned with a center pixel of the first group and a 
center pixel of the second group. 
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1 8. The image processing circuit of claim 1 6 wherein: 

the first and second groups of pixels each respectively include left, center, 
and right horizontally aligned pixels; and 
the processor is operable to, 
5 generate a first one of the direction values from the value of the right 

pixel of the first group and from the value of the left pixel of the second group, 

generate a second one of the direction values from the values of the 
right and center pixels of the first group and from the values of the left and 
center pixels of the second group, 
1 0 generate a third one of the direction values from the values of the 

center pixels of the first and second groups; 

generate a fourth one of the direction values from the values of the left 
and center pixels of the first group and from the values of the right and center 
pixels of the second group, and 
1 5 generate a fifth one of the direction values from the value of the left 

pixel of the first group and from the value of the right pixel of the second 
group. 

19. The image processing circuit of claim 1 6 wherein the processor is 

20 operable to generate the value for the filler pixel from the pixel values from which the 
processor generates the smallest one of the direction values. 

20. The image processing circuit of claim 16 wherein the processor is 
operable to generate the value for the filler pixel equal to the average of the pixel 

25 values from which the processor generates the smallest one of the direction values. 

21 . The image processing circuit of claim 1 6 wherein: 

the first and second groups of pixels each respectively include three 
horizontally aligned pixels; 

30 the pixels of the first group are vertically aligned with the respective pixels of 

the second group; 

the filler pixel is vertically aligned with a center pixel of the first group and a 
center pixel of the second group; and 
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the processor is operable to generate the value of the filler pixel equal to an 
average of the values of the center pixels if all of the direction values are greater 
than a predetermined threshold. 

22. The image processing circuit of claim 16 wherein the processor is 
operable to: 

generate the value for the filler pixel equal to the average of the pixel values 
from which the processor generates the smallest one of the direction values if the 
smallest direction value is less than a predetermined threshold; and 

generate the value for the filler pixel equal to the average of predetermined 
ones of the pixel values if the smallest direction value is greater than the 
predetermined threshold. 

23. The image processing circuit of claim 16 wherein the processor is 
operable to generate the direction values by calculating respective differences 
between pixel values in the first set and pixel values in the second set 

24. A method, comprising: 

generating a first pixel-value component from a value of an original pixel in an 
first original video image; 

generating a second pixel-value component from a value of an original pixel in 
a second original video image; 

generating a value of a filler pixel from the first and second pixel-value 
components; and 

generating a resulting video image by combining the filler pixel and the first 
original video image. 

25. The method of claim 24 wherein the second original video image 
follows the first original video image in a sequence of original video images. 

26. The method of claim 24 wherein: 

the first original video image comprises an original field having a polarity; and 
the generating the resulting video image comprises, 
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generating a filler field that includes the filler pixel and that has a 
polarity opposite to the polarity of the original field, and 
combining the original and filler fields. 

27. The method of claim 24 wherein: 

the first original video image comprises a first original field having a polarity 
and including the pixel of the first original video image; 

the second original video image comprises a second original field having the 
same polarity as the first original field and including the pixel of the second original 
video image; and 

the generating the resulting video image comprises, 

generating a filler field that includes the filler pixel and that has a 
polarity opposite to the polarities of the first and second original fields, and 
combining the filler field and the first original field. 

28. The method of claim 24 wherein: 

the generating the first pixel-value component comprises generating the first 
pixel-value component equal to the value of the original pixel of the first original 
video image; and 

the generating the second pixel-value component comprises generating the 
second pixel-value component equal to the value of the original pixel of the second 
original video image. 

29. The method of claim 24, further comprising: 
weighting the first and second pixel-value components; and 

wherein the generating the value of the filler pixel comprises generating the 
value of the filler pixel from the weighted first and second pixel-value components. 

30. The method of claim 24, further comprising: 

generating a motion value from the values of the pixels of the first and second 
original video images; 

generating from the motion value first and second weighting factors; 
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generating a weighted first pixel-value component by combining the first 
weighting factor and the first pixel-value component; 

generating a weighted second pixel-value component by combining the 
second weighting factor and the second pixel-value component; and 

wherein the generating the value of the filler pixel comprises generating the 
value of the filler pixel from the weighted first and second pixel-value components. 

31 . A method, comprising: 

generating a motion value for a first filler video image from a value of a pixel in 

F 

a first original video image and a value of a pixel in a second original video image; 
and 

causing the motion value to indicate motion for a predetermined number of 
filler video images following the first filler video image if the motion value indicates 
motion for the first filler video image. 

32. The method of claim 31 wherein the generating the motion value 
comprises generating the motion value equal to a difference between the values of 
the pixels of the first and second original video images. 

33. The method of claim 31 wherein the predetermined number equals 

five. 

34. The method of claim 31 wherein the causing comprises maintaining the 
motion value equal to a nonzero number for the predetermined number of filler video 
images following the first filler video image if the motion value equals a nonzero 
number to indicate motion for the first filler video image. 

35. The method of claim 31 wherein the generating comprises: 
generating a raw motion value for the first filler video image from the values of 

the pixels of the first and second original video images; and 

filtering the raw motion value to generate the motion value. 
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36. The method of claim 31 wherein: 

the first original video image and the original video images following the first 
original video image compose a sequence of original video images that includes the 
second original video image and a third original video image located between the 
5 first and second original video images; 

the first original video image comprises a first original video field having a 
polarity; 

the second original video image comprises a second original video field 
having the same polarity as the first original video field; and 
10 the third original video image comprises a third original video field having a 

polarity opposite to the polarities of the first and second original video fields. 

37. A method, comprising: 

generating direction values for a filler pixel from the values of first and second 
15 groups of pixels disposed in a video image, the filler pixel disposed in the video 
image between the first and second groups of pixels; and 

generating a value for the filler pixel based on the direction values. 

38. The method of claim 37 wherein: 

20 the first and second groups of pixels each respectively include three 

horizontally aligned pixels; 

the pixels of the first group are vertically aligned with the respective pixels of 
the second group; and 

the filler pixel is vertically aligned with a center pixel of the first group and a 
25 center pixel of the second group. 

39. The method of claim 37 wherein: 

the first and second groups of pixels each respectively include left, center, 
and right horizontally aligned pixels; and 
30 generating the direction values comprises, 

generating a first one of the direction values from the value of the right 
pixel of the first group and from the value of the left pixel of the second group, 
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generating a second one of the direction values from the values of the 
right and center pixels of the first group and from the values of the left and 
center pixels of the second group, 

generating a third one of the direction values from the values of the 
center pixels of the first and second groups, 

generating a fourth one of the direction values from the values of the 
left and center pixels of the first group and from the values of the right and 
center pixels of the second group, and 

generating a fifth one of the direction values from the value of the left 
pixel of the first group and from the value of the right pixel of the second 
group. 

40. The method of claim 37 wherein the generating the value for the filler 
pixel comprises generating the value for the filler pixel from the pixel values used to 
generate the smallest one of the direction values. 

41 . The method of claim 37 wherein the generating the value for the filler 
pixel comprises generating the value for the filler pixel equal to the average of the 
pixel values used to generate the smallest one of the direction values. 

42. The method of claim 37 wherein: 

the first and second groups of pixels each respectively include three 
horizontally aligned pixels; 

the pixels of the first group are vertically aligned with the respective pixels of 
the second group; 

the filler pixel is vertically aligned with a center pixel of the first group and a 
center pixel of the second group; and 

the generating the value of the filler pixel comprises generating the value of 
the filler pixel equal to an average of the values of the center pixels if all of the 
direction values are greater than a predetermined threshold. 
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43. The method of claim 37 wherein the generating the value for the filler 
pixel comprises: 

generating the value for the filler pixel equal to the average of the pixel values 
used to generate the smallest one of the direction values if the smallest direction 
5 value is less than a predetermined threshold; and 

generating the value for the filler pixel equal to the average of predetermined 
ones of the pixel values if the smallest direction value is greater than the 
predetermined threshold. 

1 0 44. The method of claim 37 wherein the generating the direction values 

comprises calculating respective differences between pixel values in the first set and 
pixel values in the second set. 
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ABSTRACT 



An image processing circuit includes a processor that receives a value of an 
original pixel of an original first video image and a value of an original pixel of an 
original second video image. The processor generates a first pixel-value component 
from the value of the original pixel of the first original video image, and generates a 

5 second pixel-value component from the value of the original pixel in the original 
second video image. From the first and second pixel-value components, the 
processor generates a value of a filler pixel, and combines the filler pixel and the 
original first video image to generate a resulting video image. One can use such an 
image processing circuit to generate a filler video field from an original video field 

1 0 and to merge the filler and original fields to generate a resulting video frame. Such 
an image processing circuit often uses less memory and detects inter-field motion 
more accurately than prior image processing circuits. Another aspect of the 
invention distinguishes thin lines from edges more accurately, and thus often 
produces fewer visual artifacts, than prior image processing circuits. 
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